It is often desirable in photography to separate the background of an image from the foreground object. For example, photographic studios, professional photographers, and others performing commercial portrait work (collectively referred to herein as “photographers”) often take pictures of humans and/or still life objects and are asked to deliver images of the subjects without the original background and/or with a different background. The request may be for advertising purposes, uniformity of a plurality of images (e.g., for yearbook or identification card pictures), and the like.
In the past, several methods have been employed to remove the background from the foreground subject in an image. A first prior method utilizes various tools in software packages such as Adobe Photoshop (by Adobe) and Paint Shop Pro (by Corel). These tools are often very labor intensive and generally comprise using erase functions and lasso style tools. In the former, the edge of the foreground object is located by hand, and the background is erased using the tool. In the latter, a tool finds edges automatically when the object is selected. However, the tool is often imprecise due to contrast and color issues between the foreground object and background. Therefore, these tools suffer drawbacks being labor intensive and imprecise, especially in cases where the foreground and background colors cannot be selected in advance. At least one prior art reference has termed these types of systems as brute force techniques.
Another method of determining the background using these types of software packages can be performed manually by splitting the image into the RGB channels. The image with the highest contrast between the foreground object and background can be selected, and then a mask can be created manually through a series of steps. The mask is used together with the original image to eliminate the background. However, this process suffers a drawback in that it is not automated and depends on a manual selection of a highest contrast channel and creation of the mask.
Chroma key replacement for background removal has also been performed by utilizing a monochromatic background. Typically green screens are used with human subjects in order to provide a contrast with skin colors. An automated system strips away portions of the image pixel by pixel by assessing whether the pixel has a color that lies within a preprogrammed range of colors. Several disadvantages associated with this system include inadvertently stripping away any pixels (in this example green) that are located on the foreground object of interest, stripping of border portions of the foreground object due to reflectance of the background green color onto the foreground object, and stripping of reflective objects located on the foreground object (e.g., watches, jewelry, etc.). Accordingly, chroma key replacement has a number of drawbacks when used in connection with fine photography, and especially in those instances where colors of the foreground object are not controlled and/or known in advance.
Another example of a prior system employed to eliminate backgrounds is shown in U.S. Pat. No. 6,885,767 to Howell. This system intentionally creates a background which is much brighter than the foreground object. In this manner, the differential in brightness between the foreground and background is used to discriminate between the two. The system utilizes a background with a very high degree of reflection, wherein incident light tends to reflect back along the path from which it came. A device, mounted on the front of the camera, includes a strobe, a partially silvered mirror, and a condenser lens. When the shutter opens, the light source causes intense light to impinge on the object and the reflective background. Due to the high reflectivity, the background is brighter than the foreground object. This image is then used as a mask for other shots of the object where the strobe is not triggered. Accordingly, this process uses an intensity technique, rather than a chroma technique to eliminate the background. However, this method also has several drawbacks, including the requirement of a particular reflective background and special equipment to create the desired light intensity coming from the plane of the camera lens. The method also has drawbacks if taking photographs of humans. More particularly, while the method may be suitable for photographing still life objects, if the object moves between the images, then the mask will not register properly with the image of the object in the other images.
U.S. Pat. No. 5,574,511 to Yang et al. illustrates yet another method of utilizing light intensity to create a mask. Here, two IR images with different intensities of IR illumination in the foreground and background are compared. A mask is then created which is applied to a visible light image. Here again, the method has drawbacks including requiring special equipment, such as IR illumination sources and a second camera having an IR pass filter. Further, in the case of photographing humans, movement of the subject between the shots may create a rough match for the mask. Also, the IR mask is not precise enough for fine photography.
Still another system is disclosed in published U.S. Patent Application 2003/0035061. In this system, a still life object is set on a turntable rotating at a constant velocity. A series of images are taken, in an alternating manner, with the foreground object first illuminated and then the background illuminated in a manner to create a silhouette of the foreground object. In one of the embodiments described in the publication, a background cutout unit is disclosed for combining a foreground image picture with a silhouette image to deliver a cutout image. In the cutout image, only the foreground images still appear. Accordingly, the cutout image can be combined at a later time with other background images. There are several drawbacks to the system disclosed in this publication. First, the system is employed with still life 3-D objects. Therefore, it does not take into consideration photographing objects which may move in a direction and/or manner other than the fixed rotational velocity of the turntable. Accordingly, photographing humans who may move between images is not considered. Second, the cutout mask is created with the object area cut-out. The mask is then used in a reversed mask layer subtraction process to remove the background. Also, the original image is not described as being preserved and transmitted—even though this image (and its attendant metadata) may be desired and/or used in other downstream processing.
Therefore, there is a need in the art for a method, apparatus and system which facilitates taking images of foreground objects in a manner in which the background can be determined, removed and replaced without relying on manual methods, chroma replacement, and/or other special IR cameras or equipment mounted in front of the camera to illuminate the foreground object. The invention should also overcome the drawbacks associated with foreground objects which may move and should not require special backgrounds or predetermined colors of the foreground object. Aspects of the present invention overcome these and other shortcomings of the prior art and address these needs in the art.